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About this document 


This document describes how the public were invited to be involved in giving feedback on the 
'Standardised Data on Initiatives (STARDIT): Alpha Version'(l) between September 24 th 2019 to the 
end of 2019. The feedback is then summarised into learning points and actions. These actions will be 
used to inform the co-creation of the Beta version of STARDIT, which will then be submitted for peer 
review. This report is licensed under a Creative Commons Attribution-ShareAlike 4.0 International 
Licence. This report has been written by Jack Nunn, Director of Science for All and PhD researcher at 
La Trobe University. This project is being run in partnership with the Wikipedia Journals (Wikimedia 
Foundation). More information can be found at ScienceForAII.World/STARDIT 
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Public consultation 


The 'Standardised Data on Initiatives (STARDIT): Alpha Version'(l) was published in September 2019. 
Opportunities to be involved in co-creating this version were advertised online using social media 
and shared via email to potential authors. Comments from co-authors were then incorporated into a 
series of versions, with all co-authors reviewing the final pre-print version. 

The pre-print was shared online and promoted using emails, newsletter and social media. Feedback 
from the public gathered by: 

• Emails 

• Phone and video calls 

• Online discussion forums 

• Online forms 

• Public events in London and Melbourne 

• Face-to-face conversations 

More information about the public event in London is shared in the next section. Relevant learning 
points from a Wikimedia Youth Salon is also incorporated into this report. 

In addition, Jack Nunn (Director of Science for All) worked with a number of people to complete 
STARDIT reports, in order to test how appropriate and useful the data entry was. This involved a 
series of phone and video calls, followed by exchanging versions of STARDIT reports in order to 
create finalised versions. 

Data from all these sources has been collated and organised into themes using qualitative thematic 
analysis. Event attendees were invited to ensure this report captured comments from the event. 
Further information about this data (including how it was collated and analysed) will be shared in 
the planned peer-reviewed paper 'Standardised Data on Initiatives (STARDIT): Beta Version'. 

London Event Summary 

On 1 st October 2019, Science for All facilitated the first public meeting about 'Standardised Data on 
Initiatives (STARDIT)'. The event was facilitated by Jack Nunn (Director, Science for All) and hosted by 
the University College London Institute of Education, London. 

Registration was free and open to anyone. People could join both in person and online. The 
facilitated discussion lasted three hours, with breaks. A detailed facilitation plan can be found in the 
supplementary materials. 

Learning from the discussion has been incorporated into feedback from other sources and has not 
been attributed to individuals. 
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List of attendees 

In person: 


• Jack Nunn - Director, Science for All, Strategy Liaison and Editor for the WikiJournals, 
member of the Cochrane Advocacy Advisory Group, PhD candidate at La Trobe University, 
Melbourne (Australia) 

• Sandy Oliver - Director of the Social Science Research Unit and Deputy Director of the EPPI- 
Centre, Professor of Public Policy at University College London, Editor of the journal 
'Research for All' 

• Carolyn Thompson - PhD Researcher, Institute of Zoology and University College London, 
Postgraduate Teaching Assistant and Lecturer, University College London. 

• Mick Mullane - Innovation Lead, National Institute for Health Research Digital Office 

• Jim Elliot - Public Involvement Lead, Health Research Authority (England) 

• Richard Stephens - Patient Advocate, Co-Editor-in-Chief, 'Research Involvement and 
Engagement', National Cancer Research Institute 'consumer' representative 


Online: 

• Chloe Mayeur - Sciensano (Belgium) 

• Wannes Van Hoof -Sciensano (Belgium) 

• James Ansell - Consumers Health Forum (Australia) 



The first 'STARDIT' selfie at the London event 

Left to right: Jack Nunn, Sandy Oliver, Carolyn Thompson, Mick Mullane, Jim Elliot, Richard Stephens 
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Learning points from the public consultation 


STARDIT as a project 

• The principle of standardised reporting described by STARDIT is useful across disciplines, 
this is 'unique' 

• STARDIT reports will be useful for a number of disciplines, including health research, 
environmental research, public policy, educational interventions and community arts 
projects. 

• Many people don't know who to trust and one participant noted that 'most of our decisions 
are based on trust'. STARDIT was identified as a way of sharing data that will facilitate 
people to critically appraise many kinds of data. 

• STARDIT is especially helpful for people to self-assess research and appraise it, including 
supporting informed decision making about whether to participate. 

• STARDIT was identified as a way of reporting how people were involved in defining 'shared 
purpose', including defining outcome measures (for example, answering the question 'what 
does success look like and how will we measure it') 

• STARDIT could provide an independent way for researchers and policy makers to show 
how people have been involved in co-producing it 

• STARDIT was identified as a helpful tool for international development, including planning, 
reporting and evaluating initiatives(2) 

• STARDIT was identified as a helpful tool for people planning, reporting and evaluating 
initiatives, including mapping preferences for involvement, reporting involvement and 
impacts from involving people. 

Proposed collaborative way of working 

• While the project is 'ambitious', the proposed collaborative way of working balances 
openness with efficiency 

• In order to make STARDIT happen, it was suggested to 'start small' and 'think like a start-up' 

• A number of partner organisations were suggested throughout the public consultation 
including Academic Health Science Networks (UK), The National Cancer Research Institute 
(UK), Independent Cancer Patients' Voice (UK), Clinical Trials Units (UK), Patient Focused 
Medicines Development (global), National Institute for Health Research (England), Good 
Things Foundation (UK and Australia), Google ('Scholar' team), National Institute of Health 
(USA), Patient-Centered Outcomes Research Institute (USA) and the James Lind Alliance 

Authenticity and trust 

"This is so global and so big - it comes back to trust, how do I trust the people who report data 
using STARDIT" 

STARDIT will be assessed by an editorial board and eventually, open peer review. It will use 
indicators from public domain sources. However, the root problem of authenticity and truthful 
reporting remains for all peer-reviewed data. While STARDIT provides data to facilitate critical 
appraisal, ongoing work will be required to ensure the authenticity of data. Partnering with the 
Wikidata project will ensure data is linked and machine-readable. Assigning Digital Object Identifiers 
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to STARDIT reports will ensure that versions are immutable, but that the reports themselves can be 
updated should errors or inaccuracies be discovered. 

Personal safety risks 

Risks were identified with STARDIT for people who may share information or data which might have 
legal or safety implications. For example, data provided by members of the public about illegal 
activities (such as poaching or illegal logging) might incriminate individuals or put those sharing the 
data at personal risk. Ways of ensuring data is shared in ways which balance transparency with 
personal safety need to be carefully considered. China was identified by one researcher as an 
example of a country where special attention and cultural sensitivity would be required. 

Life or death information 

STARDIT was identified as a tool which could help people critically appraise information which might 
be life-saving or potentially lethal if incorrect. As well as medical information, this also includes 
information on Wikipedia pages about things such as edible fungi and plants(3). 

STARDIT should have a transparent process for redacting information which might contribute to the 
destruction, poaching or killing of rare or threatened species, for example, not sharing detailed 
location information of rare species. 

Sharing power 

There is 'knowledge as power and powerful knowledge', STARDIT is a way of sharing both kinds of 
knowledge. Some 'power brokers' might not welcome knowledge sharing, transparency and scrutiny 
in certain areas and may actively resist attempts to share data and power. 'Power brokers' who 
might be resistant were identified as people working in government and for-profit organisations. 

Knowledge translation 

Understanding and measuring comprehension and knowledge translation are ongoing challenges in 
many disciplines. While STARDIT can report data on this, ongoing work will be required to ensure 
reporting is aligned with international best-practice. Partnering with organisations such as Cochrane 
and Campbell will help ensure the reporting tool is useful. STARDIT can report transformative 
learning as an impact, but this will require careful tailoring to each language and culture. 

Diversity and inclusion 

Ensuring the process for both involving people in the development of and for using STARDIT are 
inclusive will need continuous reassessment, potentially requiring a group of experts and advisors. In 
addition, as STARDIT is developed for languages other than English, groups of people specialising in 
linguistic and cultural diversity will need to be involved in ensuring STARDIT is appropriate, culturally 
safe and inclusive. 

Technical considerations 

Machine learning and 'artificial intelligence' could be employed to create reports. Wikidata is built 
for machine learning and provides an open, public domain and free way of sharing data that anyone 
can access, anywhere. After providing a way to host reports, multiple ways to submit them should 
be co-created. 
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Readability and plain English 

The 'Standardised Data on Initiatives (STARDIT): Alpha Version' needs to be improved for readability 
and plain English. In addition, the purpose and scope needs to be explained more clearly. Tailoring 
communication to specific disciplines should also be considered. Future versions that might be 
translated into other languages will require co-creation with language communities to ensure they 
are comprehensible to as wide an audience as possible. 

Systematic Searching 

Future versions of STARDIT after the Beta version will require a systematic review in order to ensure 
that all appropriate data sources have been consulted. As this will require a significant investment of 
time from those involved in the STARDIT project, it was agreed that at this stage of the co-creation 
process, a 'mini-review' (published in the peer-reviewed WikiJournal of Science) was an appropriate 
intermediate step to ensure the current search strategy is appropriate. 

Indigenous knowledge 

A report by Science for All written for the Wikimedia Foundation identified that there might be 
systematic, technical and cultural barriers to incorporating the knowledge of indigenous peoples into 
Wikipedia and other peer-reviewed repositories^). After additional meetings with staff from the 
Wurundjeri Woi Wurrung Cultural Fleritage Aboriginal Corporation, it was agreed that it could be 
helpful to explore using STARDIT to co-create a way for indigenous peoples to share traditional and 
oral knowledge. STARDIT could be used to transparently report who created any content containing 
the knowledge, what tasks they had, how this knowledge was shared and any relevant concepts of 
'owning' or 'property'. Members of Indigenous communities could work in partnership with the 
Wikimedia Foundation to create 'verified' users who formally represent relevant communities are 
have permission to share and verify knowledge (including stories, beliefs, medicine). 

The report concluded that a detailed piece of research needs to be commissioned (potentially by the 
Wikimedia Foundation) to explore concepts of 'intellectual property' and 'owning knowledge', and 
how this respectfully interacts with the free knowledge and open access movements. Certain 
cultures have restricted, taboo or 'secret' knowledge(3). This can include culturally significant sites 
which may be at greater risk of vandalism if they are shared in the public domain. STARDIT needs to 
be co-developed with the Wikimedia Foundation and with indigenous peoples to ensure that a 
balance is struck between sharing, storing and preserving unique intangible culture, while also 
remaining sensitive to respective cultural practices and attitudes regarding 'ownership'. 


Actions 

1. Create a peer-reviewed scoping review to supplement the STARDIT beta paper, in preparation 
for a future systematic review. 

2. Science for All to pay developers for creation of STARDIT report hosting. Create a project brief 
and invite developers to apply to create a beta version of STARDIT hosting. 

3. Flost more face to face and online events in other capital cities, including Canberra and Berlin. 

4. Rewrite parts of the STARDIT paper to be clearer and in plain English. 
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Supplementary resources 

Facilitation plan for public event - 1 st October: London 

Purpose of event 

Create an opportunity for people from different disciplines to talk about standardised ways of reporting initiatives, including research, education and 
international development. 

Aims 

• Summarise what different disciplines are reporting about initiatives and how 

• Suggest a common framework for reporting (STARDIT) 

• Host a discussion about common challenges and generate ideas 


Session 

Summary 

Instructions 

Outcomes 

Timing 

Introductions 

A chance to learn who is in the room, and what they 
hope to get out of today - and what the process for the 
afternoon is 

Ask everyone in room and online 
to say what their area of expertise 
or knowledge is and why they've 
come today - Online: Facilitator 
will summarise comments from 
people 

Everyone knows 
who is in the 

room and online. 

15 

Jack Nunn, 

Director of 

Science for All 

A short presentation from Jack Nunn, Director of 

Science for All - about the learning from his recent 
projects, including his PhD about involving people in 
genomics research. A short summary of 'Science for 

All' and what led to STARDIT 



10 

Carolyn 

Thompson, 

Primatologist 

A primatologist investigating small ape decline in 

China, Vietnam and Myanmar. She's working with local 
people, using participatory action research to 
investigate the patterns and drivers of critically 



10 
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endangered gibbon decline. She will discuss how 
STARDIT could be useful for recording impacts from 
this kind of work. 




Open discussion (including input via Zoom) 

Ask people online to type thoughts 
or comments to be read out (also 
check Twitter #stardit). 

Ask people in the room to share 
initial thoughts, summarise 
comments from people online. 

People online and 
in the room have 
contributed to 
discussion 

5 

Break - mingle - tell people online to log back on in either ten minutes to hear speakers or log back on at 2:10 to join the 'Idea 

Vortex' - note Australians might be going to bed and say goodnight 

10 

Reporting the 
what, who and 
how? 

Short introduction to STARDIT with 3 different 
speakers leading discussions on how it could be used 
and improved. 




Sandy Oliver - a personal and professional perspective 
(including thoughts on journals) - lessons from 
successful reporting tools? 



7 


Richard Stevens - a personal perspective as a cancer 
patient and a professional perspective in relation to 
genomics research and on journals) 

What would make people want to use this? 



7 


Jim Elliot - a personal perspective and a professional 
perspective in relation to the work of the Health 
Research Authority. What support would people need 
to use STARDIT? 



7 

Idea vortex 

Using the 'Idea Vortex' model - a series of questions 
designed to find issues and create solutions 

Welcome back people online! 


50 
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Open discussion and break 



10 

Learning so far 

A chance for anyone to speak about what has been 
learned so far, any reflections. Also a chance to map 
who's not involved who should be moving forward - 
and who will invite them! 

Invite comments from people 
online 


15 

Agreeing tasks, 
actions and 
discussion areas 

What actions have been agreed, what tasks and areas 
for further discussion 

Explain how Loomio will be used 
going forward and how actions 
and decisions will be made 

A list of actions, 
tasks and 
decisions to be 
posted on 

Loomio. 

15 

Open discussion - tea - cake - 'networking' 


20 

Formally close event 4pm. Adjourn to nearby pub. 
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Additional Discussion Points 

These additional discussion points were used to supplement the discussion: 






Can anyone write STARDIT reports? People unaffiliated with projects? Can this be one data-line 
that contributes to a 'living report' - in other words, could people report on behalf of 
organisations (like people can write Wikipedia pages about organisations without their approval) 
o Solution could be that reports make it clear when people from the organisation have 
been involved - verified (tick like on twitter?) 

How should peer review work? 

o In the short term, it will have to be an editorial board (volunteers associated with the 
WikiJournals) - we will use the existing processes of the WikiJournals for the Alpha 
version and Beta version 

o In the longer term - post beta version - it should be an open peer review process. For 
discussion 

o Peer review needs to ask the question is there evidence/data to back claims in STARDIT 
report - does it require some kind of standard critical appraisal tool? 

In the longer term STARDIT could 'score' projects 

o STARDIT scored- a peer reviewed score for an initiative which scores it on criteria 
including 'power sharing/involvement', data sharing, dissemination and translation 
o Scoring could be based on 'is there a data source for this item' so that it is not subjective 
(Binary yes or no on indicators of involvement) 
o Scoring continually reviewed but must be future-proofed so historical scores still have 
validity and use 

o This function would likely require funding/grant etc to support infrastructure - while peer 
reviewing would be voluntary the process of editorial control and back end would need 
not-for-profit investment. 


STAR 

Dissemination 

Involvement 

Translation 

4.9 

5 

2 

3 


Can things like 'documentaries' be included - who made it, who did what, who funded it? Would 
this be a category of 'educational intervention' - allowing documentaries to actually measure 
impact 

Risk of confusion with reporting guideline: STARD http://www.equator-network.org/reporting- 
guidelines/stard/ 

o Not considered an issue by attendees 
Create API for other journals etc to use with their site? Create badge 
Partners get accredited to improve participation and recruitment 
Citing Aboriginal stories - create STARDIT report for story? 
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